Particle filtering for real-time camera localisation

Real-time camera localisation is becoming increasingly important in wearable computing, particularly for applications such as augmented reality. These systems involve human interaction and therefore movement of the camera exhibits an intrinsic level of unpredictability which makes operational robustness an important consideration. This thesis investigates the potential of particle-filtering as a robust solution to the problem of determining the six degree of freedom pose (position and orientation) of a hand-held camera in real-time. The ability of the particle filter to represent non-Gaussian and multi-modal statistics have contributed to increased robustness in other areas of computer vision. Here we examine whether these benefits can be garnered in the context of real-time camera localisation. We determine the common causes of failure for real-time hand-held systems and argue that topdown Bayesian filtering is the most sensible approach to real-time localisation. Such failures are often caused by rapid, unpredictable, or erratic motions such as severe camera shake and prolonged occlusions. We are able to demonstrate that the particle filter performs significantly better than the unscented Kalman filter in these scenarios. Perhaps more importantly, we demonstrate that continuous Gaussian likelihoods traditionally applied in particle filtering can be out performed by a robust family of ‘inlier/outlier’ likelihoods. Initially we consider the problem of localising the camera using known environments in two commonly used tracking modes. Firstly, using 3-D point features identified via corner detection and normalised cross correlation. Secondly, using 3-D line models and edge detection. Subsequently we go on to examine whether the robust localisation algorithm can be extended to simultaneous localisation and mapping operation so that point maps of the environment can be built on-line. We discover that it is not straight forward to build consistent and accurate maps whilst retaining robustness to erratic motions.